Field of the Invention
This invention relates to the fields of microscopic imaging of large specimens with particular emphasis on brightfield and fluorescence imaging. Applications include imaging tissue specimens, genetic microarrays, protein arrays, tissue arrays, cells and cell populations, biochips, arrays of biomolecules, detection of nanoparticles, photoluminescence imaging of semiconductor materials and devices, and many others. More particularly, this invention relates to an instrument and method for scanning a large microscope specimen in brightfield or fluorescence with a scanning optical microscope and tilting the specimen about a scan direction during a scan to maintain focus along a length of each scan as the scan proceeds.
Description of the Prior Art
Several technologies are used for imaging large specimens at high resolution. Tiling microscopes record an image of a small area of the specimen using a digital camera (usually a CCD camera), the specimen is moved with a computer-controlled microscope stage to image an adjacent area, an image of the adjacent area is recorded, the stage is moved again to the next area, and so on until a number of image tiles have been recorded that together cover the whole area of the specimen. Images of each area (image tiles) are recorded when the stage is stationary, after waiting long enough for vibrations from the moving stage to dissipate, and using an exposure time that is sufficient to record the fluorescence images. These image tiles can be butted together, or overlapped and stitched using computer stitching algorithms, to form one image of the entire specimen. Such images may contain tiling artifacts caused by different focus positions for adjacent tiles. For large specimens, thousands of tiles may be required to image the entire specimen, requiring many changes in focus which increase the chances of tiling artifacts.
Strip scanning instruments are often used for imaging large specimens on microscope slides. In these instruments infinity-corrected microscope optics are used, with a high Numerical Aperture (high NA) microscope objective and a tube lens of the appropriate focal length to focus an image of the specimen directly onto a CCD or CMOS linear array sensor or TDI sensor, and with the correct magnification to match the resolution of the microscope objective with the detector pixel size for maximum magnification in the digitized image {as described in “Choosing Objective Lenses: The Importance of Numerical Aperture and Magnification in Digital Optical Microscopy”, David W. Piston, Biol. Bull. 195, 1-4 (1998)}. A linear CCD detector array with 1000 or 2000 pixels is often used, and three separate linear detectors with appropriate filters to pass red, green and blue light are used for RGB brightfield imaging. A high Numerical Aperture 20× microscope objective is often used, with a 1 mm field of view. The sample is moved at constant speed in the direction perpendicular to the long dimension of the linear detector array to scan a narrow strip across a microscope slide. The entire slide can be imaged by imaging repeated strips and buffing them together to create the final image. Another version of this technology uses TDI (Time Delay and Integration) array sensors which increase both sensitivity and imaging speed. In both of these instruments, exposure is varied by changing illumination intensity and/or scan speed.
Such a microscope is shown in FIG. 1 (Prior Art). A tissue specimen 100 (or other specimen to be imaged) mounted on microscope slide 101 is illuminated from below by illumination source 110. Light passing through the specimen is collected by infinity-corrected microscope objective 115 which is focused on the specimen by piezo positioner 120. The microscope objective 115 and tube lens 125 form a real image of the specimen on linear detector array 130. An image of the specimen is collected by moving the microscope slide at constant speed in scan direction 102 along the Y direction using motorized stage 105 in a direction perpendicular to the long dimension of the detector array 130, combining a sequence of equally-spaced line images from the array to construct an image of one strip across the specimen. Strips are then assembled to form a complete image of the specimen.
For brightfield imaging, most strip-scanning instruments illuminate the specimen from below, and detect the image in transmission using a sensor placed above the specimen. In brightfield, signal strength is high, and red, green and blue channels are often detected simultaneously with separate linear detector arrays to produce a colour image.
A prior art strip-scanning microscope for fluorescence imaging is shown in FIG. 2. A tissue specimen 200 (or other fluorescent specimen to be imaged) mounted on microscope slide 101 is illuminated from above by illumination source 201. In fluorescence imaging the illumination source is usually mounted above the specimen (epifluorescence) so that the intense illumination light that passes through the specimen is not mixed with the weaker fluorescence emission from the specimen, as it would be if the illumination source were below the specimen. Several different optical combinations can be used for epifluorescence illumination—including illumination light that is injected into the microscope tube between the microscope objective and the tube lens, using a dichroic beamsplitter to reflect it down through the microscope objective and onto the specimen. In addition, a narrow wavelength band is chosen for the illumination light to match the absorption peak of the fluorophore in use. Fluorescence emitted by the specimen is collected by infinity-corrected microscope objective 115 which is focused on the specimen by piezo positioner 120. Emission filter 205 is chosen to reject light at the illumination wavelength and to pass the emission band of the fluorophore in use. The microscope objective 115 and tube lens 125 form a real image of the specimen on a TDI detector array 210. An image of the specimen is collected by moving the microscope slide at constant speed in scan direction 102 along the y direction using motorized stage 105 in a direction perpendicular to the long dimension of the detector array 210, combining a sequence of equally-spaced, time-integrated line images from the array to construct an image of one strip across the specimen. Strips are then assembled to form a complete image of the specimen. When a CCD-based TDI array is used, each line image stored in memory is the result of integrating the charge generated in all of the previous lines of the array while the scan proceeds, and thus has both increased signal/noise and amplitude (due to increased exposure time) when compared to the result if a linear array detector were used.
A description of strip scanning instruments, using either linear arrays or TDI arrays, is given in US Patent Application Publication #US2009/0141126 A1 (“Fully Automatic Rapid Microscope Slide Scanner”, by Dirk Soenksen).
When either linear arrays or TDI arrays are used for scanning a tissue specimen, focus is maintained along the scan strip by moving microscope objective 115 with piezo positioner 120. A focus map for each strip is created before scanning with measurements at several positions along the strip and focus is maintained by the piezo positioner in accordance with the focus map; or automatic focus is achieved during scanning using a separate detector or focus-measuring device. One measurement of best focus position for autofocusing a point scanner (or one using a linear array detector) was described in “Autofocusing for wide field-of-view laser scanning imaging systems”, G. Li, S. Damaskinos & A. Dixon, Scanning 28(2), 74-75 (2006). This paper describes the use of an X-Z image acquired at each of several focus points on the specimen to produce a best focus position by segmenting the X-Z image along X and calculating a best focus position for each segment. The result of a best linear fit for these focus positions is used as the line of best focus. In the Y direction, the best focus is determined by a best linear fit to focus positions calculated for various Y locations. Spatial-domain intensity-gradient-based solutions were found to work better than spatial-frequency-domain-based solutions.
If the specimen is not flat, or the specimen is tilted about the scan direction, proper focus may not be achieved across the whole width of the strip. In addition, focus at the edge of adjacent strips may be different, making it difficult to stitch image strips together to assemble a complete image of the specimen without focus mismatch at the edge of strips. These problems are made worse when magnification is increased (which decreases depth of field) and when the width of the scan strip on the specimen is increased.
FIG. 3 shows one embodiment of a prior art confocal scanning laser macroscope, as described in U.S. Pat. No. 5,760,951. In this embodiment, the incoming collimated laser beam 302 from laser 300 passes through a beam expander (comprised of lens 304 and lens 306), and is expanded to match the diameter of entrance pupil 312 of laser scan lens 314 (note—entrance pupil 312 as indicated on the figure simply indicates the position of the entrance pupil. A real stop is not placed at this position). Scanning mirror 310 deflects the beam in the X direction. Laser scan lens 314 focuses the beam to spot 316 on (or inside) specimen 318, mounted on microscope slide 101, and light reflected from or emitted by the specimen is collected by laser scan lens 314, descanned by scanning mirror 310, and partially reflected by beamsplitter 308 into a confocal detection arm comprised of laser rejection filter 330, lens 332, pinhole 334, and detector 336. Detector 336 is located behind pinhole 334. Light reflected back from focused spot 316 on specimen 318 passes through pinhole 334 and is detected, but light from any other point in the specimen runs into the edges of the pinhole and is not detected. The scan mirror is computer-controlled to raster the focused spot across the specimen. At the same time, microscope slide 101, which is mounted on a computer-controlled, motor-driven scanning stage 105, moves slowly in the Y direction. The combination of rapid beam scanning across the specimen while it is moved slowly in the perpendicular Y direction results in a raster-scan motion of focused-laser spot 316 across specimen 318. A computer, represented by computer screen 340, is connected to detector 336 to store and display a signal from detector 336. The computer provides means for acquiring, manipulating, displaying and storing the signal from the detector. This confocal macroscope has properties similar to those of a confocal scanning laser microscope, except that the field of view of the microscope is much smaller. Both confocal scanning laser microscopes and macroscopes are often called “spot scanners” or “point scanners”, since they both function by scanning a focused spot or point of light across the specimen in a raster scan.
The instrument shown in FIG. 3 has the ability to adjust the gain of the detector depending on the fluorescence intensity of the fluorophore, and a high-speed preview scan can be used to predict the gain required for each fluorophore before scanning the final high-resolution image (see PCT application WO 2009/137935 A1). In addition, because the laser scan lens has a wide field of view, large specimens can be scanned in a few wide strips, making it possible to scan very large specimens (up to 6×8 inches in size in one version of a commercial instrument). Best focus is measured at several positions in the scan (Y) direction, and focus is maintained dynamically during scan. In one version of this instrument, specimen tilt was measured at several positions on the specimen, and an average value was used to adjust specimen tilt using a tilting specimen stage, using the same value of tilt across the entire specimen. This combination was very effective for thin tissue specimens and low resolution imaging (1 micron pixels or larger) but higher resolution (including the use of higher numerical-aperture scan lenses) and thick specimens will require a better solution.
Several other technologies are used for fluorescence imaging of large specimens. With tiling microscopes, the image of a small area of the specimen is recorded with a digital camera (usually a CCD camera), the specimen is moved with a computer-controlled microscope stage to image an adjacent area, an image of the adjacent area is recorded, the stage is moved again to the next area, and so on until a number of image tiles have been recorded that together cover the whole area of the specimen. Images of each area (image tiles) are recorded when the stage is stationary, after waiting long enough for vibrations from the moving stage to dissipate, and using an exposure time that is sufficient to record the fluorescence images. These image tiles can be butted together, or overlapped and stitched using computer stitching algorithms, to form one image of the entire specimen.
When tiling microscopes are used for fluorescence imaging, the areas surrounding each tile and the overlapping edges of adjacent tiles are exposed twice (and the corners four times) which can bleach some fluorophores. Exposure is adjusted by changing the exposure time for each tile. If multiple fluorophores are imaged, a different exposure time is required for each, so each fluorophore requires a separate image at each tile position. Multiple exposure of the specimen for imaging multiple fluorophores can also increase bleaching of the fluorophores.